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SUMMARY  ;  -  .  t'- 

-  ‘  I 

Li  Llilanpaijei'  w«^*considerj  a  particular  clasB 
of  games  with  partial  Information.  Generalized 
subgames  are  defined.  These  subgames  give  rise  to 
functional  equations  whose  solution  permits  a 
recursive  construction  of  the  optimal  strategies.  C  ') 
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QAMES  WITH  PARTIAL  INP0RMATI01^ 

H.  B.  Scarf 
L.  S.  Shapley 

I.  INTRODUCTION 

In  this  paper  we  shall  discuss  a  particular  class  of  games 
with  partial  Information.  Tlie  cha’^acterlstlc  feature  of  the 
Information  pattern  in  these  games  Is  that  each  player  is 
Informed  of  his  opponent’s  moves  a  fixed  amount  of  time  after 
they  are  made.  More  specifically,  the  players  each  make  a 
sequence  of  choices,  a^,  a2,  ...  and  b^,  ...»  respectively, 

from  fixed  finite  sets  A^,  A^,  ...  and  ...,  In  the 

order  a^,  b^,  a^,  b^,  ...  .  The  condition  on  the  Inforniatlon 
pattern  Is  that  Player  1(2)  In  selecting  a^(b^)  Is  Informed  of 
his  opponent’s  moves  up  to  and  Including 

as  his  ovm  previous  moves.  It  Is  nece^-aary  that  k  be  positive 
and  i  nonnegative.  The  payoff  Is  defined  to  be  some  function 
of  the  two  sequences  of  choices.  A  general  theorem  is  proved 
In  [9]  which  Implies  that  for  games  uf  this  type  continuity  of 
the  payoff  Is  a  sufficient  condition  Tor  the  existence  of  a  value 
and  optimal  strategies  for  both  players. 

The  number  A-k+i— lls  defined  to  be  the  time  lag  of 
the  game.  The  case  of  perfect  Information  Is  given  by  A  •  0. 

This  case  has  received  a  considerable  amount  of  attention 
[2,  ,  and  the  purpose  of  our  paper  Is  to  generalize  sor.o  of 

the  properties  of  games  with  perfect  infoimiatlon  to  games 
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with  positive  time  lags.  Li  order  to  Illustrate  the  properties 
that  we  wish  to  generalize,  let  us  assume  for  the  moment  that 
the  payoff  function  Is  continuous.  Let  us  define  V^(a^,  ...,  b^) 
to  be  the  value  of  the  perfect— Information  game  In  which  the 
first  n  moves  of  both  players  have  been  fixed  to  be  a^,  b^,  ...,  b 
the  payoff  being  the  same  as  the  payoff  In  the  original  case. 

The  subgame  p“*operty  of  games  with  perfect  Information  Is 
exp..essed  by  the  fact  that  the  game  which  terminates  after 
b^,  and  whose  payoff  function  Is  given  by  V'*'(aj,  ...,  b^),  has 
the  same  value  as  the  original  game,  and  that  the  optimal 
strategies  in  the  terminated  game  may  be  directly  related  to 
the  optimal  strategies  of  the  original  game  [2]  . 

The  point  about  optimal  strategies  may  perhaps  be  seen  more 
clearly  If  we  briefly  describe  the  functional  equations  associated 
with  these  subgames.  These  equations  will  be  treated  In  more 
detail  In  the  body  of  the  paper.  An  example  of  the  functional 
equations  for  perfect  Infonnatlon  games  Is 


-  Max  Mm  V^(aj, . . .  , 


*n+l  ‘’n+1 


end  their  relationship  to  optimal  strategies  Is  expressed  by 
the  fact  that  if  Player  1,  when  Informed  of  the  specific  choices 
of  a^,  b^,  plays  the  choice  of  maximizes 

Nln  V‘*’(a^,  ...,  b^,  ^n+1^'  strategy  constitutes 

‘>n+l 
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an  optimal  strategy.  The  optimal  strategies  for  Player  2  are 
derived  from  a  corresponding  set  of  functional  equations  which 
have  the  form 

•  •  • 'bn'*n+l)  ■  ••'bn+l’“n+2^  • 

bn+1  ®n+2 

The  case  k  -  1,  i  •  1,  and  X  »  1  Is  a  so-called  ’’simul¬ 
taneous  game."  In  this  case  the  subgame  property  may  be 
expressed  by  the  f'lnctlonal  equations 

V(aj....,b„)  -  tax  Min  E  n(an+i)l(bn+i)V(ai,...,b^^^) 

n+1'  '^'^n+1' 

^n+1 

•  Min  Max  ... 

where  the  p's  and  q's  are  probability  distributions,  and 
V(a^,  ...»  b^^)  Is  defined  to  be  the  value  of  the  subgame  in 
which  the  first  n  moves  of  both  players  have  been  fixed,  and 
the  game  proceeds  as  a  simultaneous  game  toward  the  same  payoff. 

If  Player  1,  when  Informed  of  the  specific  choices  of  a^^,  ....  b^, 
plays  with  a  probability  distribution  which  maxi¬ 
mizes  Min  P^®n+1  ’  ***'  ^n+1^*  then  this  collection 

bp+l  ®n+l 

of  distributions,  called  a  beaavlor  strategy,  constitutes  an 
optima',  strategy.  A  similar  remark  Is  valid  for  Player  2. 

As  soon  as  wa  begin  to  discuss  the  case  in  which  the  time 
lag  Is  greater  than  one,  the  subgame  properties  no  longer  exist. 
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The  basic  reason  for  this  failure  Is  that  If  we  fix  the  Initial 
moves  of  both  players  and  only  Inform  the  players  of  the  moves 
of  their  opponents  which  they  are  entitled  to  know,  then  the 
Information  available  to  each  player  will  be  different  at  all 
times  from  that  available  to  his  opponent.  We  will  never 
arrive  at  a  situation  v/hlch  looks  like  the  beginning  of  a  new 
game,  and  oubgames  will  therefore  not  exist. 

In  order  to  clarify  this  remark,  let  us  Introduce  a  set 
of  diagrams  describing  the  different  types  of  Information 
patterns.  The  meaning  of  the  diagrams  will  be  clear  from  the 
examples.  The  diagrams  for  the  case  of  perfect  Information 
will  be 


f 


whereas  the  diagram  for  the  game  k  ■  1,  i  -  1,  and  X  ■  1  Is 


The  diagram  for  k  •  2,  i  «  1,  and  X  ■  2  Is  given  by 


A 
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The  subgaines  In  the  first  diagram  occur  after  any  Initial 
sequence  of  moves;  In  the  second  diagram  they  occur  after 
any  Initial  sequence  which  terminates  with  a  move  of  Player  2. 

It  is  easy  to  see  that  these  represent  places  In  which  both 
players  have  a  commcn  fund  of  Information,  and  the  last  diagram 
points  out  the  fact  that  in  the  game  with  time  lag  2,  there  Is 
ho  place  In  which  both  players  have  the  same  fund  of  Information. 

As  we  shall  see.  It  Is  possible  to  Jitroduce  a  collection 
of  games  associated  with  a  game  whose  time  lag  Is  greater  than 
one,  which  play  somewhat  the  same  role  as  the  subgames  described 
for  the  cases  X  ■  0,  1,  and  which  give  rise  to  more  complex 
functional  equations  than  the  ones  mentioned  above.  It  will 
also  be  true  that  every  time— lag  game  will  have  associated  with 
It  two  functional  equations  from  welch  the  optimal  strategies 
of  either  player  may  be  deduced.  We  should  point  out  that 
these  functional  equations  have  been  discussed  by  Isaacs 

[5,  Karlin  [5,  7],  and  Dublns  for  a  particular  game,  with 
time  lag  2. 

II.  THE  QKNERALIZKD  SUB0AME3  (k  -  1,  i  -  X  >  0) 

We  shall  fix  a  specific  value  of  X  >  0,  amd  consider  the 
case  k  ■■  1,  I  «  X.  It  Is  clear  that  any  other  Information 
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pattern  with  the  same  time  lag  can  be  tranaformed  into  the 
above  ^ase  by  a  renumbering  of  the  moveB  of  one  of  the  players, 
and  the  addition  of  several  vacuous  moves  at  the  begiining  of 
the  game.  We  shall  find  it  convenient,  however,  to  discuss 
dii''ferent  combinations  of  (k,  i)  with  the  same  time  lag 
separately  (Section  IV),  and  to  introduce  a  particular  set  of 
functional  relations  for  each  combination.  What  this  means, 
of  course,  is  that  any  particular  game  will  have  several  types 
of  suogamee  emd  several  sets  of  functional  relations.  In 
particular,  the  subgames  and  functional  equations  that  we 
discuss  in  this  section  (l;  -  1,  1  -  X)  will  apply  with  the 
appropriate  renumbering  to  an  arbitrary  game  with  time  lag  X. 

The  diagram  for  this  case  is  given  by 


The  generalized  subgame  tliat  we  ere  going  to  Introduce  will  be 
described  by  a  collection  of  parameters,  which  will  summarize 
the  information  available  to  both  players  at  the  beginning  of 
the  subgame.  This  information  consists  of  two  parts: 

1.  The  complete  set  of  information  that  would  be  available 
to  Player  2  after  he  makes  his  n— th  move  in  the  original  game. 
Thils  collection  of  information  which  we  denote  by  I  consists 
of  a  specification  of  the  first  n  moves  of  Player  2  and  the 
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1 


first  n  -  X  4  1  moves  of  Player  1.  As  the  above  diagram  shows, 
this  Infomatlon  would  also  be  available  to  Playur  1  at  this 


time. 


2.  A  Joint  probability  distribution  on  the  moves 


^n-X+2*  *•*'  *^n  represent  by  Pyj(‘)* 

The  diagram  for  this  subgame  Is  as  follows: 


Pn(-) 


moves  for  Player  1 
In  subgame 


D— X+1 


n— X**»2 


ri+X+1 


'n~  X+l 


•  moves  for  Player  2 
In  subgame 

A 

The  notation  b^,  etc..  Is  used  to  Indicate  that  these  are  fixed 
choices  and  are  involved  In  the  specifications  of  the  subgame. 

The  subgame  proceeds  as  follows:  The  moves  *•*'  *^n 

are  randomlz«»d  from  Pf,(*)  «uid  told  to  Player  1,  but  not  to 
Player  2.  Player  1  then  makes  a  choice  of  followed  by  a 

choice  of  b^^j  by  Player  2.  The  choice  of  occurred 

as  a  result  of  the  randomization  Is  amnounced  to  Player  2.  The 
choice  of  b^^^  Is  told  to  both  players  after  It  Is  made;  but, 
according  to  the  Information  requirements,  the  choice  of 
Is  kept  secret  from  Player  2  until  he  Is  ready  to  make  move 

^n-tX+1*  have  a  choice  of  ayj^2  ^n'»'2'  J^®P®ct-vely, 

and  Is  then  announced  to  Player  2.  This  sequence  of  moves 

proceeds  until  all  of  the  chance  moves  have  been  announced,  and 
then  continues  using  the  Information  pattern  of  the  original  game. 
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The  payoff  la  defined  to  be  the  same  payoff  as  for  the  original 
game.  When  we  have  occasion  to  refer  to  this  suogame,  It  vrlll 
be  denoted  by  0^  ■  ^n^^n*  Clearly  Oq  Is  the  original 

game. 

The  tec^inlques  of  [9]  may  be  used  to  show  that  the  game  0^ 
will  have  a  value  and  optimal  strategies  If  the  payoff  function 
Is  continuous,  and  In  this  case  It  Is  easy  to  see  thai:  the  value 
will  be  continuously  dependent  on  the  Joint  probability  distribu¬ 
tion  specifying  the  game.  TVje  next  section  of  this  pai>er  will 
be  devoted  to  a  derivation  of  the  motional  equations  associated 
with  these  subgames,  and  we  shall  assume  In  this  derivation  tnat 
the  payoff  function  Is  continuous.  Later  on  we  shall  dlscusf* 
the  relevance  of  the  functional  equation  In  other  cases. 

III.  THE  FUNCTIONAL  RELATIONS  (k  -  1,  i  -  X) 

Let  the  value  of  0^  be  denoted  by  V(l^;  Py^(*))*  Let  us 
define  a  specific  strategy  for  Player  1  In  t-h^s  game  In  whe 
following  way.  Let  him  make  his  first  move  according  to 

the  probability  distribution  ^(^^+1  **n— X-f2'  '**'  ®n^* 

Indicate  the  dependence  o^  these  moves  upon  the  result  of  the 
randomization  In  the  obvious  way.)  After  Player  2  makes  the 

A 

move  Sind  If  the  randomized  value  of  l8  denoted  by 

A 

®n— X+2'  then  Player  1  has  complete  knowledge  of  ■ 

/A  A  ^  ^  . 

...»  X+2'  ^1*  ^n+1''  0^  course,  the  other 

randomized  values  of  a.  Let  him  then  continue  his  strategy  by 

playing  an  optimal  strategy  In  the  game  Gn+i(ln+l^  ^n+l ^ ' ^n-X+2 ^ ^ » 
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where  *  l*n— X’f2^  meant  to  be  the  Joint  distribution  on 


‘n-X-fJ 


f  •  •  •  I  a 


which  Is  formed  by  combining  Py^(*)  with 


^^®n‘fl^*n-A+2'  "*'  ®n^  conditioning  this  Joint  distribution 
by  *  *n— X*f2*  ®**  what  Player  1  can  obtain  by 

using  a  strategy  of  this  form,  if  he  tells  Player  2,  at  the 
beginning  of  0^,  that  this  Is  the  strategy  he  will  be  using. 

In  this  case  the  common  fund  of  Information  after  both  players 
have  made  their  Initial  moves  In  0^,  and  after  ^® 

Player  2,  Is  precisely  and  ^  *  1  *n— X+2 ^ ^^^®  ^® 

common  fund  with  probability  P(in— X+2^  derived  from  p^(*)* 

Player  1,  of  course,  also  knows  the  other  results  of  the 
randomization.  The  way  that  we  have  chosen  Player  I's  strategy 
shows  that  he  will  get  at  least 


^^^n+1^  Pn-^1^  *  l®n-X-»-2^^ 


Player  1  cannot  determine  the  result  of  the  randomization  for 
a^^^2»  that  at  the  beginning  of  0^  he  can  guarantee  himself 
only  an  expected  value  of 


51  P^®n-X+2^^^^ni-l^  '  l^n-X+2^^  * 

A 

Again,  Player  1  cannot  dictate  the  choice  of  so  that  he 

can  only  be  sure  of 


"in  z:  P(»,>-A+2)V(In,r  Pn+1<  ' ' ®r>-X+2) >  = 


b 


n+1 
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And  finally  If  he  picks  ’***  Judiciously,  we 

can  conclude  that 


V(ln;Pn(-)) 


x(a. 


Max 


n+1 '  n— X***2 


i:P(in_X+2)V(ln+l'Pn+l<- 

. ‘n)  "n+l 


n-X+2 


))  . 


The  next  step  is  to  replace  this  inequality  by  an  equality, 
and  this  is  accomplished  by  the  following  reasoning.  Let 
x*(a^^^ i a^^^2»  •••»  initial  component  of  an  optimal 

behavior  strategy  for  Player  1  in  Q^.  Since  the  strategy  is 
optimal,  it  can  be  told  to  Player  2  without  degrading  Player  I’s 

A 

expected  return.  Let  Player  2  choose  so  as  to  minimize 


21  P(^n-X+2^^^^n-»-l^  ^^n«»-l^  ’  l®n-X*»'2^^  ' 


where  P^+i^  *  l^n+X-2^  compounded  from  Pj^(’)  wid 
^  '*^n-fl '  ®^n— X+2'  ***'  obvious  way.  Then  with  probability 

p(a 


n-X-l-2 


)  the  common  fund  of  information  available  to  both  players 


is  Now  if  Player  2  continues  his  strategy  by  pleying  an 

optimal  strategy  in  ^n+l^  ’  ^ •n-X'f2^  ^  clear  that 

he  will  prevent  Player  1  from  getting  an  expectation  greater  than 


z:p(a 


n-X+2 


)V(I 


n+1’  ^n+1 


(• 


n-X+2 


)) 
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whlch  from  the  way  that  was  chosen  Is  equal  to 

^Mln  ^n-rl  ^  ‘  I  ^n-X-»'2^  ^  * 

n+1 

Since  Player  1  wae  assumed  to  be  playing  optimally,  this  last 
quantity  must  be  no  less  than  V(l^;  Py^(*))>  we  obtain 


V(ln:P„(-))  < 


x(a 


Max 


n-»-l 


Min  i:p(a^_^^pv(l  ;p  (-la  2 
•®n>  Vl 


Combining  this  with  the  previous  Inequality,  we  obtain  the 
deslr’ed  functional  relationship. 


Theorem  1.  Let  Gq  be  a  game  with,  time  lag  X  (written  in 
the  form  k  •»  1 ,  I  -  X ) ,  which  has  a  continuous  payoff.  Let 
V(I^;  p^(*))  be  the  value  of  the  subgame  In  which  both  players* 
information  about  the  past  is  ,  . . . ,  ;  b^ ,  .  .  .  ,  ) 

and  in  which  Player  I's  previous  X  —  1  moves  are  governed  by  the 
Joint  probability  distribution  Pf^(  *  )  ■  P^^n-X+2'  '  ’  '  *  ^n^  ’ 


v(ln:p„(-)) 


Max 


- '^n+1 


Min  i:  ■  l^n-1+2)) 


whe  re 


n-X+2' 


(a 


r)-X+2 


'n-X+3' 


n 


—13— 


T 


&nd 


n-X^2* 


P<*n-X+2‘^ 


IV.  OPTIMAL  STRATBQL^S  FOR  PLAYER  1 

In  this  section,  we  shall  show  that  a  class  of  optimal 
strategies  for  Player  1  in  the  game  with  time  lag  X  can  be 
derived  from  the  functional  equations  that  we  have  established 
in  the  preceding  section.  As  before,  we  assume  that  the  game 
is  represented  in  the  form  k*l,  i«X>0.  Optimal  strategies 
for  the  case  X  *  0  are  obtained  by  the  process  outlined  in  the 
introduction . 

The  first  of  the  functional  equations  relates  the  value 
of  Qq  (the  actual  value  of  the  game  itself)  to  the  value  of 
Ol(li;Pl(  • ) )  •  Por  ^2.^  first  equation  is 

V-  Itax  Min  v(l.:p.(-)) 
x(a^) 

with  I^  ■  and  P2(‘)  -  x(a^).  Let  us  define  the  components 
of  a  behavior  strategy  for  Player  1  in  the  following  recursive 
fashion.  Let  x(a^)  be  chosen  so  as  to  maximize 
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* .  . 

Call  such  a  maximizing  distribution  x  In  general,  if 

X  (a^),  X  ...  X  b^,  ..., 

are  known,  then  for  each  -  (a^,  ...,  ^1*  *■*'  '^n^ 

we  form  the  Joint  probability  distribution  P,^(‘)  given  by  the 
following  product  of  X  -  1  factort: 


•  '^n-X+l' 


n-1* 


•••»b^l) 


x*(a 


n-X+2'“l 


A  ^  ^  \ 

**n-X-i»l^  ^l»*‘*'^n-.X-»*l^  * 


Then  x  I  »  •  •  •  >  ^n—X+l  *  *  *  * »  *  •  *  • »  b^ )  is  chosen 

equal  to  any  ^(•^n'H^*^n~X+2'  ’**'  maximizes 


6 


Min  2:p*(a 


r>-X+2 


Ci< 


*r>-X+2^^ 


n+1 


As  takes  on  all  conceivable  value j,  we  obtain 
n  ' 

^  ^‘n^l'^l'  •• •*  *n^  ^1'  •••'  ^n'* 

We  want  to  show  that  this  method  for  selecting  the  components 

of  a  behavior  strategy  for  Player  1  leade  to  an  optimal  strategy. 

« 

Suppose  that  such  a  sequence  has  been  chosen.  Let  us  define  the 
sequence  of  functions  V  (l^)  to  be  equal  to  V(I^;  pI( •  ))•  They 
have  the  property  that 


n 


n 


I.  Min  "22  ^•n-)i+2'^l’'“’®n-X+l'  *^1’ '  ’ ' ’^n-X+l^'^  ^^n+l'  “ 

•’n+l 


and 


1 

i 
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III.  lln  V*(I  )  -  M(a,  b) 
n  " 

uniformly,  where  M  Is  the  payoff  function. 

Property  I  Is  a  direct  consequence  of  the  definitions. 
Property  II  follows  from  the  application  of  the  Initial  functional 
equation.  Property  III  Is  a  direct  consequence  of  the  continuity 
of  the  payoff  function,  which  Implies  that  the  values  of  the 
subgames  approach  the  payoff  function  for  large  fixed  Initial 
segments. 

Now  let  us  suppose  that  the  strategy  Is  played  against 

an  arbitrary  mixed  strategy  for  Player  2,  which  Is  represented 
In  behavior  strategy  form  by  the  sequence  of  conditional  distribu¬ 
tions  y('3rj+l^*l'  ’**'  ^1*  *’*'  '^n^*  strategies 

give  rise  to  a  measure  on  the  space  of  all  sequences  of  a's  and 
b's  with  the  property  that 


A, 


prob  (a^, . . . ,a^,b^, . .  .,b^)  -  x  (a^) 
(a^|a^, .  .  . 

y(b^ia^,...,a^^;b^,...,b^^)  , 


. . .  x 


and  the  functions  V(l^)  become  a  sequence  of  random  variables. 

► 


Then 
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. ^n> 

^  prob(4^, , . . 

^n-X-f2'  •  *  *  '^n-fl  *  *  *  *'^n^ 


i 


n-M 


which  in  turn  Is  equal  to 


*  „  A 


J2  *  ^*n-X>2'*l'***** 


ry-X+1 


) 


n-X+2 

A 


A  A 

btoauae  not  depend  on  ...,  Property  I 

*/  . 

tells  us  i;hat  this  last  expression  Is  not  less  than  V  (1^^), 
and  we  obtain 


E(V*(l„+l)|I„)  >  V*(IJ  . 

If  we  Integrate  out  the  conditioning  variables  and  apply 
Property  II,  we  obtain 

E(V*(l^^j))  2  V  ; 

and  applying  Property  III  yields 


K(H)  2  V  , 
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whlch  tells  us  that  our  strategy  Is  optlnval. 

There  may  be  some  question  at  this  point  as  to  which 
optimal  strategies  of  Player  1  are  obtained  from  the  functional 
equation  by  the  procedure  outlined  above.  It  Is  quite  easy  to 
give  e.'iamples  In  which  not  all  of  Player  I'c  optimal  strategies 
are  obtained  In  this  way.  It  Is  true,  but  we  shall  not  prove 
It  at  this  point,  that  the  class  of  strategies  obtained  from 
the  functional  equation  will  Include  the  class  of  "best" 
strategies  for  Player  1  [8,  p.  84^  (if  we  disregard  those 
portions  of  a  strategy  that  refer  to  situations  of  measure 
zero) . 

Theorem  2.  If  the  components  of  a  behavior  s*  rategy  for 
Player  1  are  chosen  recursively  In  the  way  outlined  above,  this 
strategy  is  optimal . 

V.  OPTIMAL  STRATE0IE3  FOR  PLAYER  2 

To  obtain  optimal  strategies  for  Player  2,  the  gaime  must 
be  represented  In  the  form  k-A  +  1,  i-O.  The  diagram  for 
the  n— th  subgame  In  this  case  Is 
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/A  A  ^  ^ 

The  game  Is  specified  by  ■  (a^,  b^,  ..., 

and  Qj^C*)  ■  prob  (b^__^^2*  _ »  notice  that  the  first 

move  In  this  subgame  Is  made  by  Player  2.  If  we  denote  Its 
value  by  V(l^;  q^(‘))»  the  functional  equation  Is 


V(I„;q„(-)) 


y(b 


Hln  -Max  2:  q ( b^.x+g > : ^n+l < '  I  Va+2 >  > 

n+l'°n-X-f2**  •  •  ®n+2 


and  by  using  the  same  techniques  as  In  Section  IV,  It  Is  possible 
to  compute  an  optimal  strategy  for  Pliyor  2  In  a  recursive  fashion 
from  this  functional  equation. 

VI.  THE  QSNERALIZED  SUBOAMES  (OTHER  VALUES  OF  k  ^  i) 

In  this  section  wo  consider  the  representation  of  our  game 
with  time  lag  X  for  general  values  of  k  and  i  (k  >  1,  I  >  0, 

X  ■  k  +  I  —  1).  For  each  value  of  (k,  i)  there  will  be  two 
classes  of  subgames,  depending  on  which  player  moves  first. 

These  will  be  generalizations  of  either  the  games  discussed 
in  Section  III  or  those  discussed  In  Section  V.  In  what  follows 
we  shall  restrict  our  attention  to  the  former. 

The  diagram  for  the  n— th  subgame  Is  given  by 
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n-/+l 


A  ^  K 

'’n-k+lv _ _ nj 


moves  for  Player  1  In  the  subgame 

_ A _ 


-n+1 


u^±I.. 


^ovea  for  Player  2  in  the  subgame 


This  subgame  la  described,  first  of  all,  by  the  fund  of 
Information  known  to  both  players  after  Player  2  has  made  his 
n— th  move.  In  this  case  It  will  be  a  specification  of  Player 
I's  first  n  -  .4  +  1  moves,  and  Player  2*s  first  n  —  k  1  moves, 
say  (a^,  •••,  *  ^n*  given 

an  arbitrary  pair  of  Joint  probability  distributions  P^(*)  • 
prob(a^^^2»  ®^n^  and  q^(*)  • 

game  will  be  denoted  by  proceeds 

as  follows:  The  moves  a^  ...,  a^  are  randomized  from  p  (•) 

n— j+2'  n  "^n 

and  told  to  Player  1  but  not  to  Player  2.  Simultaneously,  the 
moves  b^  ...,  b^  are  randomized  from  Qp(')  and  told  to 

Player  2  but  not  to  Player  1.  They  then  proceed  as  they  would 
in  the  original  game,  with  the  same  payoff  M(a,  b) .  We  still 
assume  that  this  payoff  Is  continuous,  so  that  the  general 
theorem  of  [9]  applies  and  the  game  has  a  valuf:: ,  which  we 
denote  by  V(I^;  p^{-),  qj^(*))*  As  before,  there  exists  a 
sequence  of  functional  relations.  Tliey  take  the  form 
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V(ln;Pn(-),qn(-)) 


-  Max  Min 

5Z  P^*n-i+2^^^^iv-k+2^'^^^fl^Pn+l^  *  l^n-i+2^'‘^n+l^  ’  I^n-i-f2^  ^ 

••  Mil)  Max  ... 

The  proof  la  quite  similar  to  the  proof  given  in  Section  III 
and  we  shall  not  repeat  it  here. 

We  would  like  to  indicate  the  major  difference  between  the 
functional  equations  in  this  case  and  the  functional  equations 
that  were  discussed  in  Sections  III  and  IV.  In  those  sections 
we  showed  how  an  optimal  strategy  for  Player  1  could  be  computed 
recursively  from  the  functional  equation.  The  corresponding 
procedure  for  the  present  case  would  be  the  following:  Suppose 
that  x*(a^)  ...  x*(a^|aj,  ...,  a^^^,  b^,  ...»  b^)  have  been 
computed.  Then  for  any  I^  -  (a^,  ...,  fij#  •••#  ^n-k+3^' 

we  would  consider  a  game  ®n^^n^’^n^  ’  defined 

by 


Pn(-)  -  x*(a 


A  A  A 

n-i+2'®l* • *  * '*n-i+l'^l' • • • '^n-X+1^ 


...  X  (^nla;^,...,aj^__^,b^,...,b^)  , 


and  for  some  q  (•)  which  for  the  moment  we  leave  undefined. 

n 

Then  ^(*^n4'l^*^n-i*f2'  ^n^  would  be  chosen  so  as  to  maximize 
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vfh  Ih  b  ^  r  Vi+2) 


n 


‘  l^n-k+2^^  * 


It  ie  clear  that  this  diolce  would  depend  on  qj^(*)»  *^0  would 

therefore  only  be  able  to  show  that  the  strategy  we  have  chosen 
1b  optimal  against  a  particular  choice  of  Player  2's  strategy. 

This  means  that  In  order  to  obtain  an  optimal  strategy  for 
Player  1  In  an  arbitrary  (k,  I)  Representation  (X  >  0),  we  must 
transform  this  representation  Into  k  ■  1,  I  •  X  by  renumbering 
the  moves  of  one  of  the  players  and  the  addition  of  several 
vacuous  moves  at  the  beginning  of  the  game,  and  then  apply  the 
method  of  Theorem  2.  To  obtain  optimal  strategies  for  Player  2, 
we  must  transform  Into  k  ■  X  +  1,  1*0. 

VII.  SOME  RHMARK3 

► 

In  our  discussion  we  have  consistently  assumed  that  the 
payoff  function  Is  continuous.  This  has  permitted  us  to  say 
that  each  subgame  under  discussion  has  both  a  value  and  optimal 
strategy  for  either  player,  and  there  Is  at  least  one  point  In 
the  proof  that  we  have  given  for  the  validity  of  the  functional 
equation  In  which  the  existence  of  optimal  strategies  was 
specifically  used.  There  are  other  conditions  under  which 
our  subgames  may  be  shown  to  have  a  value.  For  example,  as 
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iB  shown  In  [93*  IP  the  ptyofr  runcMon  lu  lippot-  (lown  )  neml— 
continuous,  then  each  subgame  has  a  value  an!  optlm/tl  n’r'ategles 
exist  for  Player  1  (2).  The  question  ai-lnes  af^  to  wt,eilier*  the 
functional  equations  relating  the  values  ol’  iheae  nul  games  are 
still  valid.  It  can  be  shown  that  a  modi  hi cattnn  ol  the  argument 
of  Section  III  yields  this  same  functional  equation,  with  Max  Min 
replaced  by  Max  Inf  (Sup  Min).  It  Is  also  true  that  oi)timal 
BtrateglkJ  for  the  player  who  has  them  In  the  8eml-<'ontlnuous 
case  can  also  be  generated  by  m^ins  of  the  functional  equations. 

On  the  other  hand,  very  little  can  be  said  about  the  other  player's 
strategies  from  the  functional  equation.  To  Illustrate  this  point, 
let  us  assume  that  the  payoff  Is  lower  semi— continuous ,  so  that 
the  maximizing  player  does  not  necessarily  have  an  optimal 
strategy.  Let  us  recursively  pick  strategies  for  Player  1,  by 
choosing  an  ^('*'^+1  ^®n— X+2'  ***’  ^n^  which  Is  6/2*^-ef fectlve 
In 

Mir,  ' 

‘’n+1 

and,  as  before,  define  V  (l^)  -  V(I^;  p^(*))*  Then  It  will 
be  true  that  against  any  strategy  for  Player*  2  we  have 

E(V*(I^^l)lIn)  2 '“'(In)  -  therefore  E(v‘(I^))  ^  V-£. 

But  lower  semi— continuity  weakens  Property  III  of  Section  V  to: 

in'  Tim  M(a,b)  >  Tim  V*(l  )  >  Um  V*(I  )  ^  M(a,b)  . 

a,b-^a,b  n-^oo  n->OD 

As  a  result,  we  can  conclude  that  E(  llm^  M(a,b))  V  -  t » 

a,b-^  a,b 

but  not  that  S(M(a,b))  ^  V  -  €. 
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Even  without  the  condltlona  of  semi-con tlnulty  on  the 
peyoff  function,  It  Is  still  meaningful  to  talk  about  the 
functional  equations.  Without  any  conditions  on  the  payoff 
function,  if  we  can  find  a  solution  of  the  functional  equations 


)) 


(a 


Max 


n+l '  n-X-»-2' 


’*n) 


.Min  2:  P(“n-X+2)''(^n+l'Pn-n( 


n-X+2 


)) 


n-fl 


with  the  property  that  V(I^;p^(*))  ->M(a,  b)  (say  boundedly), 
then  the  strategy  for  Player  1,  which  Is  generated  recursively 
from  these  equations,  will  guarantee  Player  1  at  least  V(Iq) 
against  any  strategy  of  Player  2. 


EXAMPLE 

It  nay  bo  instructive  to  show  how  the  above  techniques  can 
be  applied  to  the  celebrated  ”boinber-battleshlp”  game  to  yield 
functional  equations.  (References  [5],  [5]/  [7].)  We  shall 

do  this  in  two  ways,  according  to  the  two  arrangements 


Case  I 


I 


Case  11 


I 


n’¥2 


(a 


1 


k  -  3,  i  -  0 

a  d  a.2  vacuous) 
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Th«  will  eaoh  b«  0  or  1;  the  will  each  be  0,  1,  2,  or 
0  (paaa).  The  flret  a^  0  le  Interpreted  as  a  prediction  that 


"n  *  ‘’nt'l  "  ‘n  *’h-S  Vl  "  “n 

(in  oaae  I).  (in  Case  II). 

The  payoff  ia  1  to  the  a-player  for  a  correct  prediction,  0  for 
an  incorrect  pi^edlction  or  no  prediction.  This  payoff  ia  lower 
aemi-oontinuoua,  ao  optimal  atrategiea  are  assured  only  for  the 
b-player.  (Thia  fomulation  follows  Blackwell  [l].) 

In  Case  I  we  observe  that  the  generalized  subgane  * 

is  trivial  if  any  a^  f^0in  1^^.  On  the  other  hand, 

•nd  ajl^i  Pjn(*))  AP*  completely  isomorphic  provided  that 
Pj^  •  Pjjj,  bjj  •  b^,  and  all  a^^,  a^  are  0  (l.e.,  the  earlier 
^i'  ^i  matter).  Hence  we  may  write 


where  b^j  and  -  Pj,(«).  Moreover,  syiinetry  tells  us  that 

fo(*0'  *1'  ^2^  "  ^1(^2*  *1'  *0^’  Hence  the  functional  equations 
of  Theorem  1,  n  ^  1,  reduce  to  the  single  equation: 


(1) 


*  Max  Min 
u,v,w 


tfQ(u,v,w)  +  X 

tfQ(w,v,u)  y 


where  t-1  —  x  —  y  —  z  and  u,  v,  w  are  restricted  to  be  non- 
negative  with  sum  ^  1.  The  first  equation  (n  >  0)  becomes 
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for  tho  valuo  of  the  game.  Equation  (l)  is  the  sanr  aa  equation 

(34)  In  laaaoa*  paper  [6]  and  forma  the  baala  of  hla  analyala 

of  € -optimal  strategies  for  the  a-player.  The  unique  "ideal* 

(locally  optimal)  strategy  Is  given  by  •  0  —  l.e., 

never  predict  —  and  Is  clearly  not  optlnal. 

In  case  II,  On(lj,J  trivial  If  any  a^^  0 

In  I^,  while  the  other  games  are  entirely  Independent  of  all 

b.  In  I  .  Thus,  we  have 
in 

V„(I„;<1„(-))  -  8(it)  If  n  ^  1, 


whtpe  X  »  0  ^  X  ^  1.  SjTBMtry  glv«»  us  g(x)  •  g(l  -  x). 

Hence,  the  functional  equations  of  Section  V  reduce  to 


(2) 


S(x)  •  Min 

O^u^l 
0  iv<l 


Naxs 


XU  (0) 

x(l-u)  (l-x)v  (1) 

(l^)(l-v)  (2) 

xg(u)  (l-nx)g(v)  (0) 


with 


V  -  Vq  -  Min  g(x). 


No  Max  appears  because  a^  Is  an  automatic  pass  In  this  case. 
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ThlB  functional  equation  has  been  studied  exhaustively.  It 
develops  that  a  minimizing  x*  can  be  chosen  for  (5)  with  the 

property  that,  If  we  set  x  •  x  In  (2),  the  mlnlmuLi  Is  achieved 

•  • 

at  u  -  X  ,  V  -  1  —  X  .  This  makes  an  optimal  strategy  for  the 

b— player  extremely  easy  to  describe:  always  choose 

•  « 

with  probability  x  ,  b^_^^  •  1  ~  b^  with  probability  11  —  x  . 

There  la  reason  to  believe  that  this  phenomenon  will  not  occur 
in  other  related  cases,  such  as  the  X  •  3  version  of  the  present 
example,  but  the  theory  remains  obscure  on  this  point. 


) 
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